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METHOD AND SYSTEM FOR ADAPTIVE 
NETWORK SECURITY USING 
INTELLIGENT PACKET ANALYSIS 

CROSS-REFERENCE TO RELATED 5 
APPLICATIONS 

This application is related to U.S. patent application Ser. 
No. 09/223,072 entitled "Domain Mapping Method and 
System", filed Dec. 29, 1998, pending, and U.S. patent 
application Ser. No. 09/222,414 entitled "Method and Sys- 
tem for Adaptive Network Security Using Network Vulner- 
ability Assessment", filed Dec. 29, 1998, pending. 

TECHNICAL FIELD OF THE INVENTION 

The present invention relates in general to computer 
network security and, more particularly, to a system and 
method for adaptive network security using intelligent 
packet analysis. 



20 



BACKGROUND OF THE INVENTION 



Network security products such as intrusion detection 
systems (ID systems) and firewalls can use a passive filter- 
ing technique to detect policy violations and patterns of 
misuse upon networks to which the Security products are 
coupled. The passive filtering technique usually comprises 
monitoring traffic upon the network for packets of data. A 
signature analysis or pattern matching algorithm is used 
upon the packets, wherein the packets are compared to 
"attack signatures", or signatures of known policy violations 
or patterns of misuse. 

In order to properly detect policy violations and patterns 
of misuse, security products often must place the packets of 
data in contexts relevant to such connection criteria as space, 
time, and event. Space is usually defined in terms of a 
source-destination connection at the port level. Time is 
defined as the amount of time to continue associating 
packets for the type of connection defined by the source- 
destination connection. Event is defined as a type of 
connection, which in turn defines the types of policy and 
misuse signatures that can occur with each packet. As the 
size of a network expands, there arc greater numbers of 
connections which leads to greater numbers of lookups and 
comparisons that must be performed by the security product. 

Two problems are associated with conventional security 
products. First, conventional security products have insuf- 
ficient information to self-configure for reliable detection of 
policy violations and patterns of misuse. For example, 
conventional security products have no mechanism to reh- 50 
ably ascertain network information of the network to which 
the security product is coupled. This leads to such disad- 
vantages such as being unable to accurately predict the effect 
of a particular packet upon a destination device. 
Furthermore, a conventional security product has no mecha- 55 
nism to ascertain the network topology and thus cannot 
predict if a certain packet will reach its intended destination. 
Such a lack of network information compromises the secu- 
rity product's ability to detect such attacks such as insertion 
attacks, evasion attacks and denial of service attacks. Some 
of these problems with conventional security products are 
documented by Ptacek and Newsham, Insertion, Evasion, 
and Denial of Service: Eluding Network Intrusion 
Detection, Secure Networks Incorporated, January 1998. 

A second problem associated with conventional security 65 
products is the result of scarcity of processor and memory 
resources. Conventional security products may begin to drop 



packets and shut down certain tasks in an unpredictable 
fashion once the system depletes its memory or processor 
resources. As the size of a network grows, such a failure 
becomes more likely, as the greater the number of connec- 
tions onto the network requires a greater number of lookups 
and comparisons performed by the Security product. 
Additionally, an increase in number and complexity of the 
types of misuse the security product is required to detect can 
fiirther degrade performance. An increase in traffic flow 
further drains a security product's resources. As a result, 
conventional ID systems cannot operate effectively at high 
network bandwidth utilization. 

Some conventional systems have attempted to achieve 
performance gains by decreasing the number of misuse 
signatures the security product monitors. Fewer signatures 
translate into fewer memory comparisons for each packet 
that flows through the security product. However, such a 
solution^ makes a network more vulnerable to attacks. 

Other conventional systems rely on the user to enumerate 
the network information, such as the types of operating 
systems and applications running on the protected network. 
These systems then disable certain misuse signatures 
accordingly. 

Such a conventional solution, however, introduces addi- 
tional problems. For example, if the user provides an inac- 
curate assessment of the network, then incorrect signatures 
may be disabled, meaning that undetected policy violations 
and network attacks are possible. Additionally, networks are 
rarely stable environments and the addition or deletion of 
devices or services can make the original network informa- 
tion supplied by the user inaccurate. 

A further disadvantage of such conventional security 
products is that they are not designed to function in an 
environment wherein the traffic exceeds their memory or 
processor capacity. Such conventional systems, when con- 
fronted with traffic that exceeds their capacity, may start 
dropping packets and degrade performance in an unpredict- 
able fashion. This can lead to an unknown security posture 
or profile, which can leave a network more vulnerable to 
undetected attacks. 

SUMMARY OF THE INVENTION 

In accordance with the present invention, a method and 
system for network security based upon intelligent packet 
analysis are disclosed that provide significant advantages 
over prior developed network security systems. 

According to one aspect of the present invention, a 
method comprises monitoring network data traffic. The 
network data traffic is analyzed to assess network informa- 
tion. A plurality of analysis tasks are prioritized based upon 
the network information. The analysis tasks are to be per- 
formed on the monitored network data traffic in order to 
identify attacks upon the network 

In one embodiment, the method further includes moni- 
toring a processor utilization and disabling a particular 
analysis task based upon an assigned priority of the particu- 
lar analysis task if the processor utilization exceeds a first 
defined threshold. 

In an additional embodiment, the method further includes 
re -enabling a disabled analysis task if the processor utiliza- 
tion drops below a second defined threshold. 

According to another aspect of the invention, a system for 
adaptive network security using intelligent packet analysis, 
comprises an analysts engine coupled to a network. The 
analysis engine analyzes network data traffic to assess net- 



11/19/2003, EAST Version: 1.4.1 



us 6,4' 

3 

work informatioQ. A protocol engine is coupled to the 
network, and the protocol engine performs a plurality of 
protocol analyses on the network traffic to identify attacks 
upon the network. A signature engine is also coupled to the 
network. The signature engine compares the network trafiSc 
to a plurality of attack signatures to identify attacks upon the 
network. Apriority engine is coupled to the analysis engine, 
the protocol engine, and the signature engine. The priority 
engine is for prioritizing the plurality of protocol analyses, 
the plurality of attack signatures based upon the network 
information. 

According to another embodiment of the present 
invention, tlie priority engine can prioritize a plurality of 
system services based upon the network information. 

It is a technical advantage of the present invention that it 
can more reliably detect policy violations and patterns of 
misuse because of the use of the network information. 

It is another technical advantage of the present invention 
that it allows for the maintenance of a network map, which 
can allow for greater types of misuse patterns to be detected. 

It is a further technical advantage of the present invention 
that it allows for a reliable and predictable prioritized 
shutdown of analysis tasks and services in the event 
resources are depleted. 

It is another technical advantage of the present invention 
that effective intrusion detection can be had at higher net- 
work bandwidth utilization than conventional security sys- 
tems. 

It is another technical advantage that the present invention 
provides for adaptive network security, as the invention can 
adapt to a changing network environment and recalibrate in 
order to maintain a sufficient level of network security. 

It is an additional technical advantage that the present 
invention can create and maintain network information in a 
network map without placing additional traffic upon the 
network. 

Other technical advantages should be apparent to one of 
ordinary skill in the art in view of the specification, claims, 
and drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

A more complete understanding of the present invention 
and advantages thereof may be acquired by referring to the 
following description taken in conjunction with the accom- 
panying drawings, in which like reference numbers indicate 
like features, and wherein: 

FIG. 1 is a flow diagram of various embodiments of a 
method of operation of a system for adaptive network 
security; 

FIG. 2 is a block diagram of one embodiment of a network 
envirorunent that includes a system for adaptive network 
security using intelligent packet analysis; 

FIG. 3 is a block diagram showing network information 
of one embodiment of a network that includes a system for 
adaptive network security using intelligent packet analysis; 

FIG. 4 is a flow diagram of one embodiment of a method 
for adaptive network security using intelligent packet analy- 
sis; and 

FIGS. 5 A, 5B, and 5C are block diagrams of embodi- 
ments of a prioritized task list, a prioritized attack signature 
list, and a prioritized system services list, respectively. 

DETAILED DESCRIPTION OF THE 
INVENTION 

FIG. 1 is a flow diagram of various embodiments of a 
method of operation of a system for adaptive network 
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security. An ID System is one such security system that 
could benefit from the adaptive network security system of 
the present invention. 

In the method of FIG. 1, network information is acquired 
5 at step 1. Network information can comprise, for example, 
the devices, operating systems, and services available on a . 
network. 

In the embodiments of FIG. 1, such network information 
can be gathered by an active process 2, a passive process 4, 
or a query process 3. Active process 2 can include port scans, 
pinging, and other active methods performed on devices 
coupled to the network, as well as monitoring responses 
(such as banners) sent in response to such active methods. 
One such active process is described in the related U.S. 
patent application Sen No. 09/222,414 entitled "Method and 
System for Adaptive Network Security Using Network 
Vulnerability Assessment", filed Dec. 29, 1998, pending. 
Query process 3 can comprise sending a query to a domain 
mapping service, wherein the domain mapping service 
maintains a compilation of network information. Such a 
domain mapping service can respond to such a request by 
sending the network information to a source of the request. 
Such a query system is described in the patent listed above, 
as well as described in U.S. patent application Ser. No. 
09/223,072, entitled "Domain Mapping Method and 
System", filed Dec. 29, 1998, pending. 

The third alternative to. acquire network information is 
passive process 4, Passive process 4 allows a security device 
using the present invention to acquire network information 
without placing additional traffic on the network. One such 
passive process is an intelligent packet analysis. A method 
and system for adaptive network security using intelligent 
packet analysis is described more fully below. 
35 Once network information is acquired, an analysis at step 
5 is performed. For example, a network map 6 can be created 
to compile the network information. At step 7, a priority task 
is performed using the analysis of the network information 
at step 5, For example, an ID system using such a method 
can configure itself to perform high priority tasks based 
upon potential vulnerabilities of the network, as identified 
by the analysis at step 15. 

The performance of steps 1, 5, and 7 can occur in one or 
more devices coupled to a network. For example, processes 
45 performing such tasks could be distributed among several 
devices in order to preserve processing resources. 
Alternatively, the processes performing such tasks could be 
integrated into a single device, such as an ID system, router, 
or firewall. 

50 FIG. 2 is a block diagram of one embodiment of a network 
environment that includes a system for adaptive network 
security using intelligent packet analysis according to the 
present invention. As shown, the network environment can 
comprise devices that form an internal network, protection 

55 for the internal network, and an external network. The 
internal network, indicated generally at 10, can comprise a 
plurality of workstations 12 coupled to a network backbone 
14. Network backbone 14 can comprise, for example, an 
Ethernet, FDDI, token ring, or other type of physical media 

60 type. Protection for intemal network 10 can be provided by 
firewall 16 and a router 18 which are coupled to network 
backbone 14. Router 18 serves as a gateway between 
internal network 10 and an external network 30. External 
network 30 can be, for example, the Internet or other public 

65 network. Firewall 16 can serve to limit external access to 
resources in internal network 10 and protect these internal 
resources from unauthorized use. 
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Internal network 10 further comprises network security 
system 20 coupled to network backbone 14. Although FIG. 
2 displays network security system 20 coupled to internal 
network 10 through network backbone 14, those skilled in 
the art will recognize network security system 20 can couple s 
to internal network 10 in other ways, such as through 
workstation 12. Network security system 20 comprises a 
protocol engine 24 coupled to network backbone 14. An 
analysis engine 22 and a signature engine 26 each couple to 
protocol engine 24. Analysis engine 22 is further coupled to 
network map 28. Signature engine 26 is coupled to attack 
signatures 30. A priority engine 32 is coupled to network 
map 28, protocol engine 24 and signature engine 26. Pro- 
tocol engine 24 and signature engine 26 each also couple to 
a storage 36. 

In the embodiment of FIG. 2, network security system 20 
is coupled directly to network backbone 14 "inside" internal 
network 10, Such a configuration is typical, for example, of 
an intrusion detection system. However, those skilled in the 
art will recognize that network security system 20 can be 
coupled to a network in other configurations. For example, 
network security system 20 could be incorporated into 
another device located on internal network 10, such as 
firewall 16 or router 18. Alternatively, as further shown in 
FIG. 2, network security system 20 could be coupled outside ^5 
internal network 10, such as between firewall 16 and router 
18, or outside router 18. It should be understood that 
different placement of network security system 20 will affect 
the its operation, as different placement exposes network 
security system 20 to different traffic on the network. 3Q 

Network security system 20 can comprise, for example, 
software code executing on a computing device such as a 
SUN or INTEL based workstation. Network map 28 and 
attack signatures 30 can comprise data stored in memory or 
fixed storage on the workstation or other device in which 35 
network security system 20 resides. Storage 36 can comprise 
memory or fixed storage that is the same as or separate from 
the memory upon which network map 28 and/or attack 
signatures 30 reside. Alternatively, some or all of storage 36 
and the data that comprises network map 28 and attack 4Q 
signatures 30 could reside in fixed storage remote from the 
location of network security system 20. Similarly, analysis 
engine 22 could comprise software code executing remotely 
from the device upon which network security system 20 
resides. One example of such an alternate configuration, for 45 
example, is shown in FIG. 2 as a domain mapping system 39 . 
and network map 41. 

In operation, devices such as workstations 12 can com- 
municate over network backbone 14. Workstations 12 can 
further communicate with external network 30 via network 50 
backbone 14 and router 18. As mentioned above, firewall 16 
is intended to prevent unauthorized access from external 
network 30 to devices coupled to internal network 10. 
However, firewall 16 may not capable of preventing all 
unauthorized access. As used with respect to this 55 
application, "attack" is used to describe any type of unau- 
thorized access, policy violation, or pattern of misuse. 

Further in operation, network security system 20 is oper- 
able to detect attacks upon internal network 10, Network 
security system 20 accomplishes this by monitoring traffic 60 
on network backbone 14 and performing analysis tasks upon 
the monitored traffic in the context of network information 
discovered from internal network 10. In the embodiment of 
FIG. 2, protocol engine 24 monitors the traffic for packets of 
data, analysis engine 22 analyzes the packets to assess 65 
network information, while protocol engine 24 and signature 
engine 26 perform analysis tasks upon the monitored traffic. 
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Analysis engine 22 couples to protocol engine 24 and can 
an analyze the traffic to assess network information. For 
example, analysis engine 22 co\dd monitor the types of 
services being accessed on certain devices by analyzing the 
content of packets addressed to the device in question. The 
services could be deduced by maintaining service state 
tables based upon the types of packets that pass through the 
monitored network. 

Additionally, in the embodiment. of FIG. 2, analysis 
engine 22 is operable to analyze the network information to 
identify potential vulnerabilities of internal network 10. For 
example, analysis engine 22 could perform a rules-driven 
assessment on the network information that analysis engine 
22 has detected. Such an assessment could comprise, for 
example, a portion of the rules-driven multi-phase network 
vulnerability assessment described in U.S. patent applica- 
tion Ser No. 09/107,964, entitled "System and Method for 
Rules-Driven Multi-Phase Network Vulnerability 
Assessment," filed Jun. 30, 1998, the disclosure of which is 
herein incorporated by reference. 

Analysis engine 22 can further create a network map 28 
which can include such network information discovered by 
analysis engine 22. Network map 28 can comprise, for 
example, a multi-dimensional database with a real-time data 
insertion, as described in U.S. patent application Ser. No. 
09/107,790, entitled "System and Method for Real-Time 
Insertion of Data Into a Multi-Dimensional Database for 
Network Intrusion Detection and Vulnerability 
Assessment," filed Jun. 30, 1998, pending, the disclosure of 
which is incorporated herein by reference. 

Further in operation, protocol engine 24 performs a plu- 
rality of protocol analyses upon monitored traffic on network 
backbone 14 in order to detect attacks upon the network. 
Attacks upon the network, as mentioned above, are defined 
herein to include unauthorized accesses, policy violations, 
and patterns of misuse. Protocol engine 24 can perform, for 
example, the following protocol analyses upon monitored 
traffic on network backbone 14: checksum verification (IP, 
TCP, UDP, ICMP, etc.), IP fragment reassembly, TCP stream 
reassembly, protocol verification (such as insuring the IP 
header length is correct and the TCP data gram is not 
truncated), and timeout calculations. 

Signature engine 26 is coupled to protocol engine 24 and 
can perform further analysis tasks on the network data traffic 
in order to detect attacks upon internal network 10. Signa- 
ture engine 26 compares the packets of monitored network 
data traffic with attack signatures 30. Attack signatures 30 
can comprise, for example, a rules-based hierarchy of traffic 
signatures of known policy violations. Signature engine 26 
can compare packets from the network data traffic with such 
attack signatures 30 such that policy violations can be 
discovered. 

Further in operation, priority engine 32 uses the network 
information maintained in network map 28 to prioritize the 
analysis tasks performed by the protocol engine 24 and the 
signature engine 26. For example, priority engine 32 could 
determine a likelihood of success of a particular attack upon 
the network based upon the network information. Priority 
engine 32 could then prioritize the protocol analysis per- 
formed by protocol engine 24 that is intended to detect that 
particular attack. Likewise, priority engine 32 could priori- 
tize attack signatures 30 based upon the network information 
in network map 28 according to the Likelihood of success of 
each attack associated with each attack signature 40. In one 
embodiment, priority engine 32 could compile a prioritized 
task list comprising a list of all such analysis tasks, ranked 
by an assigned priority to each task. 
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Additionally, priority engine 32 could prioritize system such packet can be "captured'* at step 100. At step 104, the 

services performed by network security system 20. Such traffic or packet is analyzed to discover network information 

system services could include, for example, IP logging, and count attributes. Such network information can 

traflGc logging, alarm notificatioas, and communications comprise, for example, devices coupled to the network, 
among others. 5 operating systems running on the devices, and services 

Further in operation, priority engine 32 can monitor a f ^"''''l* °" ^ ^ '^'X'"^' a step could be accomplished, 

memory utilization of memory resources and a processor example by maintaining service state tables based on the 

utiUzation of processor resources. If the processor utilization '^Pf °^ P"^^^^ P*^, ^^^h Uie network. 

A 4 J J *u u ij • i ^1 J- ui At step 108, potential vulnerabihties associated with 

exceeds a first denned threshold, priority cngmc 32 disables j . ^ i j * .1. * 1 ^ . ■ j t- 

, . . , . , . . ^ 1*1 ij in devices coupled to the network are determined. For 

an analysis task. As used herem, analysis task could com- 1 *u • * 1 j 1 u j 

, » \, ^ , example, this step could comprise a rules-based comparison 
pnse a protocol analysis performed by protocol engine 24 or ^^^^^^ discovered network information, and known 
a particular attack signature 40 as used by signature engine problems associated with networks that contain such con- 
26. Priority engme 32 can make the disable decision based figurations. Such a comparison is described, for example, in 
upon an assigned priority of each task, as discussed previ- y § ^ent application Ser. No. 09/107,964. At step 112, the 
ously. Then, as processor utihzaUon dropped below a second 15 discovered network information is used to create and main- 
defined threshold, pnonty engine 32 could reenable the a ^^^^ ^ network map. 

disabled analysis task. Similarly, priority engine 32 could a* * u ui e ^- 1 ^* i 

^. , i f ^ . At step 116, a probable success of a particular attack upon 

disable a particular analysis task or system service it * - jt / * i u 

^. i , ... -, J J . ij the network is determined. In order to make such a 

memory utilization exceeded a third denned threshold, or j , . ^ 1 • ^ * j • .u 

, / , 1 , , . 1 .^^ .1- . 1 determination, the network information stored in the net- 

reenable a disabled analysis task 11 memory utilization drops 20 i- j * u *u * 1 i no j 

i-ijriji.ij work map can be applied to both protocol analyses 118 and 

below a lourth derined threshold. ,.1 . . r 1 * 1 i 

attack signatures 120. For example, protocol analyses can 

The results of the protocol analysis provided by protocol comprise checksum verification, protocol verification, IP 

engine 24 and signature analysis provided by signature fragment reassembly, and TCP stream reassembly, as dis- 

engine 26 are recorded in storage 36. The results could then mussed above. Each of the above protocol analyses 118 can 

be made available, for example, to another process or a intended to discover a particular type of attack. Depend- 

system administrator. jjig yp^jj ^jjg network information stored in the network map, 

Network security system 20 is adaptive because it can ft can be determined whether or not an attack that is 

configure or reconfigure by prioritizing the protocol discovered by such a protocol analysis has a certain prob- 

analyses,the attack signatures, or its system services accord- ability of success. Likewise, each of the attack signatures 

ing to changes in network information. 120 are designed to detect a particular type of attack upon 

FIG. 3 is a block diagram showing network information the network. The network information contained in network 

of one embodiment of a network that includes a system for map can assist in determining the probability of success of 

adaptive network security using network intelligent packet each potential attack as defined by its associated attack 
analysis. This diagram also shows the dimensionality of a ^5 signature. 

network and its devices in terms of device types 70, oper- At step 124, the analysis tasks and system services are 
a ting systems 74, services 78 and potential vulnerabihties prioritizeid. Protocol analyses 118 and attack signatures 120 
80. Such dimensionality, for example, could comprise the are assigned a priority based upon the determined probabil- 
network information discovered by network security system ity of success performed at step 116. System services 121 are 
20 and stored in an associated network map. Internal net- prioritizedbaseduponalevelofcriticalityof each service as 
work 10 of FIG. 3 comprises numerous devices, including can be determined from the network information. System 
router 28, firewall 16, web server 50, workstations 52, 56, 60 services 121 include services performed by the security 
and 62, file server 54, printer 64, and terminal server 58. device such as: IP logging, event logging, or alarm sound- 
Each of these devices is coupled to network backbone 14. ing. The prioritizing of such services is based upon the 
Similar to FIG. 2, network security system 20 is coupled to network information, as the network information determines 
network backbone 14. the level of necessity of each system service. 

In operation, as discussed with respect to FIG. 2 network At step 128, system monitoring is performed. The system 

security system 20 monitors network data traffic and ana- monitoring is performed to discover a memory utilization of 

lyzes the irafi&c to assess network information of internal memory resources at step 130, a processor utiUzation of 
network 10. As further discussed with respect to FIG. 2, 50 processor resources at step 132, and an overall system 

network security system 20 can discover network informa- bandwidth 133. System bandwidth 133 might be particularly 

tion such as device types 70, operating systems 74, and affected under a denial of service attack, for example, 

services 78 on internal network 10. Additionally, network At step 134, an enable/disable function is performed. For 

security system 20 of FIG. 3 can make an assessment of example, if the processor utiUzation has exceeded a particu- 
potential vulnerabihties 80 associated with each device on 55 lar threshold, for example 90%. a particular analysis task 

internal network 10. (either a protocol analysis 118 or a particular attack signa- 

All such network information can be incorporated into ture 120) can be disabled. Alternatively or additionally, a 

network map 28 (FIG. 2). Priority engine 32, further as particular system service 121 may be disabled. This particu- 

discussed with respect to FIG. 2, can use the information in lar analysis task can be reenabled if the processor utilization 
network map 28 to prioritize the analysis tasks to be per- 60 drops below a second defined threshold, for example 85%. 

formed on monitored trafSc by protocol engine 24 and Similariy, if the memory utiUzation exceeds a third defined 

signature engine 26. threshold, a particular analysis task can be disabled. If the 

FTG. 4 is a flow diagram of one embodiment of a method memory utiUzation subsequently drops below a fourth 

for adaptive network security using intelligent packet analy- defined threshold, the particular analysis task can be recn- 
sis. At step 100, network data traflBc is monitored on the 65 abled. 

network. Network data trafSc can comprise, for example, By enabling or disabUng system services 121 at step 134, 

packets exchanged between devices on the network. Each the security system implementing such functionaUty can 
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adapt to a changing network environment. The systems that 
a security system performs can be referred to as a configu- 
ration of the security system. As the network information 
drives the services performed by the security system, the 
security system is able to configure and reconfigure itself as 5 
the network dynamics dictate. 

At step 138 it is determined if the analyzing for network 
information should be repeated. If so, the method returns to 
step 100 to discover updated network information, and the 
method is repeated. By obtaining updated network 
information, and then repeating the prioritizing steps using 
the updated network information, the method can adapt to a 
changing network environment. 

FIGS. 5A, 5B, and 5C are block diagrams of embodi- 
ments of a prioritized task list, a prioritized attack signature 
Ust, and a prioritized system services list, respectively. In 
FIG. 5A, a prioritized task list, indicated at 144, comprises 
a plurality of analysis tasks 148. Prioritized task list 144 
includes both types of analysis tasks: protocol analyses and 
signature analysis 150. The analysis tasks, as discussed 
above, are intended to identify particular attacks upon the 
network and can include both protocol analysis and com- 
parisons between network trafBc and known attack signa- 
tures. In the embodiment of FIG. 5A, the analysis tasks have 
been prioritized from the least important (TCP checksimi) to 
the most important (signature analysis 150), according to the 
network information of a particular network. 

FIG. 5 B is a prioritized attack signature list 150, and FIG. 
5C is a prioritized system services Ust 152. Similar to 30 
prioritized task list 144, prioritized attack signature list 150 
and prioritized system services list 152 are created based 
upon network information gathered from a network that the 
security system is coupled to. 

For example, the priority engine 32 (FIG. 2) may con- 35 
struct prioritized task hst 144. Then, if memory or processor 
resources are depleted, the priority engine can disable cer- 
tain analysis tasks 148, beginning with the least important, 
until the memory or processor utilization is at a safe oper- 
ating threshold. It should be understood, however, that the 40 
present invention contemplates that in some circumstances, 
analysis tasks could be disabled or re -enabled "out of order", 
that is, not according to an assigned priority. Such could 
occur, for example, upon a user intervention or upon the 
presentation of a particular attack. For example, if an 45 
attacker launches an IP fragment DOS attack against the 
network security system, the system should detect the attack 
through the IP fragment reassembly analysis task. The 
system then may disable IP fragment reassembly for some or 
all other fragments and issue an alarm about the attack. 50 

Further in operation, once it is determined that an attack 
signature must be disabled, as shown in FIG. 5B, low 
priority attack signatures can be disabled before higher 
priority attack signatures. Additionally, as shown in FIG. 5C, 
system services prioritized and disabled accordingly. 

The present invention further contemplates that in some 
instances it may be desirous to disable certain tasks regard- 
less of memory or processor utilization. Such an instance 
could occur, for example, if a user wished to disable all 
attack signatures made irrelevant by the network informa- 
tion discovered on the network. 

Although the present invention has been described in 
detail, it should be understood that various changes, substi- 
tutions and alterations can be made thereto without depart- 65 
ing from the spirit and scope of the invention as defined by 
the appended claims. 



55 



What is claimed is: 

1. A computer implemented method for adaptive network 
security using intelligent packet analysis, comprising: 

monitoring network data traffic; 
analyzing the network data traffic to assess network 
. information; 

prioritizing a plurality of analysis tasks based upon the 
network information, the analysis tasks to be performed 
on the monitored network data traffic in order to 
identify attacks upon the network; 

wherein the plurality of analysis tasks includes a plurality 
of comparisons between the monitored network data 
traffic and a plurality of attack signatures; and 

disabling a particular attack signature based upon an 
assigned priority of the particular attack signature. 

2. The method of claim 1, further comprising disabling a 
particular analysis task based upon an assigned priority of 
the particular analysis task. 

3. The method of claim 2, further comprising: 
monitoring a processor utilization; and 

performing the disabling step if the processor utilization 
exceeds a first defined threshold. 

4. The method of claim 3, further comprising re-enabhng 
the particular analysis task if the processor utilization drops 
below a second defined threshold. 

5. The method of claim 2, further comprising: 
monitoring memory utilization; and 

performing the disabling step if the memory utilization 
exceeds a third defined threshold. 

6. The method of claim 5, further comprising re-enabling 
the particular analysis task if the memory utilization drops 
below a fourth defined threshold. 

7. The method of claim 1, wherein the prioritizing step 
comprises: 

determining a probable success of a particular attack upon 
the network based upon the network information; and 

assigning a priority to the particular analysis task intended 
to detect the particular attack. 

8. The method of claim 1, further comprising: 
comparing the network information to existing network 

information to determine updated network information; 
and 

repeating the prioritizing step using the updated network 
information. 

9. The method of claim 1, further comprising: 
prioritizing a plurality of system services based upon the 

network information; and 
disabling a particular system service based upon an 
assigned priority of the particular system service. 

10. The method of claim 1, wherein the analyzing step 
comprises determining a device coupled to the network. 

11. The method of claim 1, wherein the analyzing step 
comprises determining an operating system of a device 
coupled to the network. 

12. The method of claim 1, wherein the analyzing step 
comprises determining a service of a device available to the 
network. 

13. The method of claim 1, wherein the analyzing step 
further comprises identifying a potential vulnerability of a 
device on the network. 

14. The method of claim 1, further comprising maintain- 
ing the network information in a network map. 

15. The method of claim 1, wherein the plurality of 
analysis tasks includes protocol analysis on the monitored 
traffic. 
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16. The method of claim 15, wherein the plurality of 
analysis tasks includes checksum verification. 

17. The method of claim 15, wherein the plurality of 
analysis tasks includes IP fragment reassembly. 

18. The method of claim 15, wherein the plurality of 5 
analysis tasks include TCP stream reassembly, 

19. The method of claim 15, wherein the plurality of 
analysis tasks includes timeout calculations. 

20. A computer method for adaptive network security 
using intelligent packet analysis, comprising: lO 

monitoring network data traffic; 

analyzing the network data traffic to assess network 
information; 

prioritizing a plurality of protocol analyses to be per- 
formed on monitored traffic from the network, the 
protocol analyses for identifying attacks upon the net- 
work; 

monitoring a processor utilization; 

monitoring memory utilization; 20 
disabling a particular protocol analysis based upon an 

assigned priority if the processor utilization exceeds a 

first defined threshold; and 
disabling a particular protocol analysis based upon an 

assigned priority if the memory utilization exceeds a 

third defined threshold. 

21. The method of claim 20, wherein the analyzing step 
comprises determining the existence of a device coupled to 
the network from a packet of monitored network data traffic. 

22. The method of claim 20, wherein the analyzing step 
comprises determining an operating system running on a 
device coupled to the network from the monitored network 
data traffic. 

23. The method of claim 20, wherein the analyzing step 
comprises determining a service of a device coupled to the 
network from the monitored network data traffic. 

24. The method of claim 20, further comprising identi- 
fying potential vulnerabilities of each device discovered to 
be coupled to the network. 

25. The method of claim 20, further comprising 
re-enabling a disabled protocol analysis if the processor 
utilization drops below a second defined threshold. 

26. The method of claim 20, further comprising 
re-enabling a protocol analysis if the memory utilization 
drops below a fourth defined threshold. '^^ 

27. The method of claim 20, wherein the plurality of 
protocol analyses includes checksum verification. 

28. The method of claim 20, wherein the plurality of 
protocol analyses includes IP fragment reassembly. 

29. The method of claim 20, wherein the plurality of 
protocol analyses includes TCP stream reassembly. 

30. The method of claim 20, wherein the plurality of 
protocol analyses includes timeout calculations. 

31. The method of claim 20, further comprising: 
comparing the network information to existing network 

information to determine updated network information; 
and 

updating the prioritizing step using the updated network 
information. 

32. The method of claim 20, further comprising: 
prioritizing a plurality of system services based upon the 

network information; and 
disabfing a particular system service based upon an 
assigned priority of the particular system service. 65 

33. A computer implemented method for adaptive net- 
work security using intelligent packet analysis, comprising: 
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monitoring network data traffic; 

analyzing the network data traffic to assess network 
information; 

prioritizing a plurality of comparisons between monitored 
network data traffic and a plurality attack signatures 
based upon the network information, the attack signa- 
tures for identifying attacks upon the network; 

monitoring a processor utilization; 

monitoring memory utilization; 

disabling a particular attack signature based upon an 

assigned priority if the processor utilization exceeds a 

first defined threshold; and 
disabling a particular attack signature based upon an 

assigned priority if the memory utilization exceeds a 

third defined threshold. 

34. The method of claim 33, wherein the prioritizing step 
comprises: 

determining a likelihood of success of a potential attack 
based upon the network information; and 

prioritizing an attack signature of the potential attack 
according to the determined likelihood of success. 

35. The method of claim 33, wherein the analyzing step 
comprises determining the existence of a device coupled to 
the network from monitored network data traffic. 

36. The method of claim 33, wherein the analyzing step 
comprises determining an operating system type of a device 
coupled to the network from monitored network data traffic. 

37. The method of claim 33, wherein the analyzing step 
comprises determining a service of a device coupled to the 
network from a packet monitored network data traffic. 

38. ITie method of claim 33, further comprising identi- 
fying potential vulnerabilities of each device discovered to 
be coupled to the network. 

39. The method of claim 33, further comprising 
re-enabling a disabled comparison if the processor utiliza- 
tion drops below a second defined threshold. 

40. The method of claim 33, further comprising 
re -enabling a disabled comparison if the memory utilization 
drops below a fourth defined threshold. 

41. The method of claim 33, further comprising main- 
taining the network information in a network map. 

42. The method of claim 33, further comprising: 
comparing the network information to existing network 

information to determine updated network information; 
and 

repeating the prioritizing step using the updated network 
information. 

43. The method of claim 33, further comprising: 
prioritizing a plurality of system services based upon the 

network information; and 
disabling a particular system service based upon an 
assigned priority of the particular system service . 

44. A system for adaptive network security using intelli- 
gent packet analysis, comprising: 

an analysis engine coupled to a network, the analysis 
engine for analyzing network data traffic to assess 
network information; 

a protocol engine coupled to the network, the protocol 
engine for performing a plurality of protocol analyses 
on the network data traffic to identify attacks upon the 
network; 



11/19/2003, EAST Version: 1.4.1 



us 6,499, 

13 

a signature engine coupled to the network, the signature 
engine for comparing the network data traffic to a 
plurality of attack signatures to identify attacks upon 
the network; and 

a priority engine coupled to the analysis engine, the ^ 
protocol engine, and the signature engine, the priority 
engine for prioritizing the plurality of protocol analyses 
and the plurality of attack signatures based upon the 
network information. 

45. The system of claim 44, further comprising a network 
map coupled to the analysis engine and the priority engine; 

wherein the analysis engine is operable to maintain the 
network information in the network map. 

46. The system of claim 44, wherein the priority engine is 
further operable to disable a particular analysis task based 
upon an assigned priority of the particular analysis task. 

47. The system of claim 44, wherein the priority engine is 
further operable to: 

monitor a processor utilization; and 
disable the particular analysis task if the processor utili- 
zation exceeds a first defined threshold. 

48. The system of claim 47, wherein the priority engine is 
further operable to re -enable the particular analysis task if 
the processor utilization drops below a second defined 25 
threshold. 

49. The system of claim 44, wherein the priority engine is 
further operable to: 
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monitor memory utilization; and 
disable the particular analysis task if the memory utiliza- 
tion exceeds a third defined threshold. 

50. The system of claim 49, wherein the priority engine is 
further operable to re -enable the particular analysis task if 
the memory utilization drops below a fourth defined thresh- 
old. 

51. The system claim 44, wherein the priority engine is 
further operable to: 

determine a probable success of a particular attack upon 
the network based upon the network information; and 

assign a priority to the particular analysis task intended to 
detect the particular attack. 

52. The system of claim 44, wherein the network infor- 
mation comprises: 

a device coupled to the network; 

an operating systems running on the device; and 

services available on the device. 

53. The system of claim 52, wherein the network infor- 
mation further comprises a potential vulnerability of the 
device on the network. 

54. The system of claim 44, wherein the priority engine is 
further operable to prioritize a plurality of system services 
based upon the network information. 

♦ )(t ))e « « 
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